WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress - #1014

One WHIR (multilinear) proof per block, with no epochs (the prove-and-retire / VADCOP shape). This is a second prover next to #1010's epoch-based one. #1010 stays the reference and this branch does not touch it. The branch starts from #1010's head f3d359998; compare against f3d359998 to see only this work.

Status: draft, work in progress. One binary (FAST 416, #1010's head 88b0d31 merged in, ABBA × 2): the block tree 23.20 s whole vs #1010's epoch tree 31.80 s, −8.60 s. Since then: 20.89 s whole at d9d0ac5 (FAST 424: base 16.76 + recursion 4.13), with the batched argue on by default, ops dropped after commit (M1), the lean walk and paged executor memory (E1/E4), and one prepared stack per group. e06dce5 added narrow trace storage (M4: same bytes, time-neutral, max RSS 1× 36.2 → 29.5 GiB, 4.13× now proves). 3cfffe2 added the KECCAK and ECSM split (§6: the same tables and proof bytes on every block measured so far, FAST 830). 01da99f uploads phase A's next group beside the current commit (prover-only, same bytes, base −0.60 s at 1×, FAST 832). 59e9890 lays out the streamed chunks on three threads by default (E3; prover-only, same bytes; whole block −0.48 s to 19.75 s and max RSS −1.51 GiB at 1×, FAST 838). 82f9046 re-hashes phase B's kept-top paths in parallel (prover-only, same bytes; phase B −0.96 s, whole block −0.82 s to 18.90 s at 1×, FAST 843 + 844 pooled). d169edd lets the tree's leaves execute while their artifacts are built (prover-only, same bytes; level 0 −0.24 s, whole block −0.24 s to 18.54 s at 1×, FAST 841 + 842 pooled). 70eee3e lays out the rest in waves and builds KECCAK_RND as its tables (W, lane i-m4b; same bytes; max RSS 88.9 → 79.5 GiB at 4.13×, base −0.14 s at 1×). fac261f lets the tree's first finished leaf execute during phase B (R2-ii; prover-only, same bytes; whole block −0.24 s to 18.39 s at 1×, measured before the merge with W, FAST 845 + 846 pooled). 365e3ab packs the finish's tables as they are built and commits them narrow (b2, lane i-m4b; same bytes; max RSS 78.8 → 52.1 GiB at 4.13×, 28.5 → 22.5 at 1×; base −0.36 s at 1×). 351a773 adds a device readout to the tree test (test-only), and the measurement posture moves to the card memory pool's code default, which retains freed memory: a baseline shift, not a prover change (whole block 17.91 → 17.27 s at 1×, FAST 848 + 849 pooled; every number above was taken under the old posture and stays as measured). 9a18ab5 makes the prover refuse a block over the group maximum as its groups close, with the verifier's own error (lane i-m4b; prover-only, same bytes, FAST 858). 54aae28 builds the recursion tree's node programs beside the tree, each level proving as its own nodes arrive (A; prover-only, same bytes; median whole block 242.69 → 233.80 s, −8.89 s, BIG 611; 2.66× recursion −1.22 s; 1× unchanged), and closes a latent card-permit re-entry that parked BIG 569. d107a91 proves each node of the tree as soon as its own children are proved (C; prover-only, same bytes; median recursion −1.73 s, BIG 612; 2.66× recursion −0.49 s, BIG 614 + 615; 1× unchanged). 0d61081 spills the held tables to disk when memory is short (lane i-m4b; prover-only, same bytes; #1013's store and auto policy, on by default; nothing spills at 1× or 4.13× by default, and the 1× time is unchanged within noise; BIG 616, ULTRA 006). acfd54a counts phase 4's BITWISE lookups in slices (#1013's c369843, ported by lane i-m4b; prover-only, same bytes; median whole-run VmRSS 108 → 89 GiB, phase A −9.5 s, BIG 619; 1× base −0.30 s, ULTRA 010). c55dec4 lets the tree's top execute each child's share as that child is proved, while a child is still unproved (B′; prover-only, same bytes; median recursion −0.47 s, BIG 613 / 620; 1× unchanged, where the top executes whole, ULTRA 009). 806c211 lets each node of the tree execute from its program as soon as it is emitted, with only its prove waiting for its artifacts (b′; prover-only, same bytes; 1× recursion −0.27 s, ULTRA 012; 2.66× −0.41 s, BIG 624; median unchanged). Head cbfa7d7 grinds the block's WHIR proofs at 18 bits instead of 20 and opens 114 queries instead of 112 (§7, a format change approved by Mauro 10-05; proven bits unchanged at 130.393; 1× whole block −0.27 s, ULTRA 019; median whole block −3.42 s, BIG 628). Every format change below but §6 (the port of #1013's S0a, gated FAST 830) is approved by Mauro (10-02; §7 on 10-05); the cryptography team reviews them at the end (CRYPTO-REVIEW.md rows W-0…W-6, W-8, W-9).

What it is

• prover::block_whir::prove_block_whir / verify_block_whir. The whole block is one MultiProof:
  • every table is cut into instances of at most 2^21 rows, and KECCAK_RND into 2^16-row instances;
  • memory is the monolithic PAGE argument, with no local-to-global bookend and no cross-epoch proof.
• The tables are packed into groups of at most 3 stacked polynomials (2^27 each).
  • Phase A commits each group and retires it: the codewords go, and only the top of each tree stays on the host (the bottom 4 levels are dropped).
  • The roots block: every root goes into the transcript, then (z, α, β) are drawn once.
  • Phase B proves each group on its own fork S_post ‖ g: upload again, argue (LogUp-GKR), re-encode with the NTT alone, open. The dropped tree levels are rebuilt from the queried cosets and checked against the kept nodes.
  • The bus is checked once, over every table of the block.
• Streamed phase A. The executor runs in 2^20-cycle windows feeding the shared WindowedTraceBuilder (noepoch/windowed-builder, also used by STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013). Every full chunk of CPU, MEMW_R, MEMW_A, MEMW, LOAD, LT, SHIFT and STORE is laid out and committed as soon as it exists.
• The builder is deterministic: on block 25368371 the whole-run build and two windowed builds under host contention give identical per-table digests (FAST 411; 129 tables row for row, the six HashMap-ordered tables as row multisets).

⚠ Format changes (all approved by Mauro, 10-02 (§7 on 10-05), but §6, the port of #1013's S0a; for the cryptography team's end review; ledger rows W-0…W-6, W-8, W-9 in CRYPTO-REVIEW.md)

The block is one WHIR proof (BlockWhirProof, one MultiProof) over all of the block's tables.

• Phase A commits the tables in groups as the trace is built. Each group is at most 3 stacked polynomials of 2^27, and each group is committed and retired as it closes.
• The challenges. The transcript then absorbs the statement, the groups' roots and the derived prepared roots, and draws z, α, β.
• Phase B proves each group on its own fork of the transcript (S_post ‖ g).

Every parameter below is a verifier-side constant. None is read from the proof.

1. The group partition is in the statement

What. The streamed prover packs tables into groups in the order their chunks complete. It writes the partition (per group, its tables' indices) into the statement. The transcript absorbs it before any root.

Verifier. The verifier checks:

• that the partition is exact (every table once);
• that each group of two or more tables fits within 3 stacked polynomials under the verifier's own stack cap;
• that there are at most 64 groups.

It then rebuilds every group's stack layout itself.

Security. Proven bits are unchanged.

• Each WHIR chain is 130.393 bits, and the proof minimum under the campaign's min-over-phases accounting is 128.946.
• The chain heights are capped by BLOCK_MAX_TABLE_VARS (change 3), so no partition can make a chain taller than 2^27.

Proof bytes vary from run to run. Arrival order depends on thread timing, and the hash-ordered tables' row order (LT, EQ, BYTEWISE, BRANCH, MUL, DVRM) follows a hash state that is random per process. Mauro, 10-02: "the proof not having the same bytes it's fine, they never had the same bytes anyways".

• Byte-identity tests run with LAMBDA_VM_FIXED_TRACE_HASH=1 (fixed hash keys) and LAMBDA_VM_DETERMINISTIC_GRIND=1. Under both, two processes prove the same bytes (FAST 423).
• Fixed keys let a crafted program steer many operations into one bucket, so they are a test setting, not production's.

2. Prepared openings, one stack per group (approved by Mauro, 10-02: "it's a standard technique, go ahead")

In short. This is #1010's existing prepared DECODE opening, carried into the block format. It is extended to the dense genesis pages and stacked per group.

• Costs: +0.14 s of base and −0.57 s of recursion (4.71 → 4.14 s).
• The verifier recomputes every derived root from the ELF.

What. The recursion guest cannot evaluate DECODE's five 2^20 preprocessed columns, nor the dense genesis pages (millions of rows), on its own. So each group's prepared tables (DECODE and the dense genesis pages chosen by genesis_stack_plan) have their leading preprocessed columns stacked into one commitment.

• Both sides derive this commitment from the ELF and the partition. The verifier recomputes every derived root from the ELF: verify_block_whir, and verify_block_tree's plan. The prover's own roots only shortcut the prover's side.
• Its roots are absorbed after the groups' roots, before z.
• It is opened on the group's fork after the group's own opening, each table's block at that table's point.

Negatives, on a guest with two dense genesis pages. Each is refused by the host verifier and by the recursion leaf, and admitted when the openings are skipped (the mutation):

• two groups' openings swapped;
• a block opened at another table's point.

Wrongly committed stacks are refused at the roots block: two pages swapped, a page holding another page's columns, another program's DECODE.

Cost. One stack per group costs +0.14 s of base (FAST 419; one commitment per table cost +0.49 s) and saves 0.57 s of recursion, by taking 90 k permutations out of the leaves. BlockFormat::prepared = false turns the openings off, for measurement only.

3. ECDAS split; table heights capped

What.

• ECDAS is split into tables of at most 2^17 rows, as KECCAK_RND is at 2^16 (split_ecdas).
• A scalar multiplication may straddle two tables. Its steps chain only through the Ecdas bus, keyed by the call's timestamp and the step's (round, op).
• The frame (the host verifier and the tree plan) refuses:
  • a KECCAK_RND table over 2^16 rows;
  • an ECDAS table over 2^17 rows;
  • any table over 2^27 rows (REV-JUDGE G2). A 2^30 table would otherwise make a 2^30 chain at 127.39 bits.

Why. At a median block, one ECDAS table (2^19) argues on a 12 GiB tree. At p90 (2^20) it stacks into five polynomials on a 24 GiB tree.

Gates. FAST 534 and 535: the negatives, 3 mutations caught.

4. Recursion leaf: the leaf cap (G3) and the inverse share (W1)

• G3. The tree plan refuses a group over LEAF_PERMS_CAP. With no fixed leaf count, it adds leaves while the heaviest leaf is over the cap. The block's own partition is unchanged.
• W1, on by default. Each WHIR leaf publishes its bus share Σ p/q as p · ediv(1, q), which has no satisfying assignment for q = 0 whatever p is. The former ediv(p, q) left the share free at p = q = 0 (≈ 2^-160 under GKR soundness).
  • The leaf gains one XALU op per table, so every WHIR tree id changes. The base proof is untouched.
  • Gate: FAST 536. LFM_WHIR_SHARE_INVERSE=0 keeps the former form.

5. Batched argue, on by default since d9d0ac5 (BlockFormat::argue = ArgueFormat::BATCHED; Mauro, 10-02: "batch the constraints")

What. Each group's tables are argued together on the group's fork:

• one lockstep LogUp-GKR ladder per bin, the tables packed first-fit-decreasing under 2^26 input cells;
• one front-loaded constraint sumcheck per group;
• no claim reduction for unshifted tables (every VM table).

Verifier side. The verifier derives the bins from the statement's shapes and its own cap; the variant is never read from the proof. The proof carries one BatchedArgue per group in BlockWhirProof.argues, and proof.tables is empty under this format. The recursion leaves verify the same format (lane i-batch2, N-4).

Security at the block's measured inputs (D-BATCH §3.2's formulas; re-checked at FAST 424's inputs: |T| ≤ 60 tables a group, ≤ 51 trees a bin, ≤ 2^28.87 input cells, bus messages ≤ 204 elements, N_C ≤ 413, D_max 4, n ≤ 21; L = 300):

Every term is above the WHIR phase, so the proof minimum is unchanged: 128.946 under the campaign accounting (130.393 for the WHIR fold under the calculator of record).

Measured.

• FAST 348: phase-B argue −0.685 s, whole block −0.385 s (≈ −0.65 s attributable).
• FAST 424 at the flip (d9d0ac5, A B B A, one binary): base 17.56 → 16.76 s (−0.80), whole block 21.85 → 20.89 s (−0.96), recursion −0.16 s. Every proof verified; determinism within each format 2/2.
• Peaks: BIG 402/403 found none raised at 1.0 / 1.31 / 1.80×. BIG 395 at 1.80× with ops dropped: per-table 62.39 against batched 63.12 GiB (+0.72).

The knob. ArgueFormat::PerTable (one argument per table, the format before) stays measurable, with BLOCK_WHIR_ARGUE=per-table in the real-block tests. At the per-table format, the MultiProof, the prepared openings and the partition are digest-equal to the head before the batched merge (BIG 393).

6. KECCAK and ECSM split; their heights capped (the port of #1013's S0a; landed at 3cfffe2; approved by the lead under Mauro's any-block target, Mauro to confirm)

What.

• KECCAK is split into tables of at most 2^18 rows and ECSM into tables of at most 2^17 (split_keccak, split_ecsm), as ECDAS is at 2^17.
• A row of either is one whole call (a permutation, a scalar multiplication), so a cut falls between calls. A call reaches its rounds (KECCAK_RND), its steps and scalar bits (ECDAS) and its memory only through buses keyed by its timestamp.
• The frame takes up to 2^12 tables of each and refuses a KECCAK table over 2^18 rows or an ECSM table over 2^17. Every other verifier keeps both to one table.

Why. A keccak-heavy block at the gas limit makes about 2^21 permutation calls. One KECCAK table that tall stacks into eight polynomials of 2^27, against a group budget of three. At its cap each table fits one polynomial.

Bytes. Every block measured so far has KECCAK ≤ 2^17 rows and ECSM ≤ 2^11, so it builds the same tables and the same proof (FAST 830: digest 07d1bd43… unchanged on block 25368371).

Gates. FAST 830: KECCAK and ECSM forced into 4 tables each on block 25368371 prove and verify, base and tree; the count and height negatives; mutation C caught.

7. WHIR grinding 20 → 18 bits, queries 112 → 114 (landed at cbfa7d7; approved by Mauro, 10-05)

What.

• Each WHIR chain grinds 18 bits instead of 20 before each round's queries, and opens 114 queries instead of 112 to buy the two bits back. At every block height (15 to 27 variables) Q = num_queries(2, rounds, 128, 18).

• One site, multilinear_prove::chain_config_under. Every side of the block reaches it through BlockFormat::chain_config:

  • the prover's group chains and prepared openings;
  • verify_block_whir;
  • the plan the recursion leaves' in-guest verifier is emitted from.

  The statement absorbs Q and the grind bits.

• Scope: the block's WHIR proofs only. The recursion tree's own leaf, node and top proofs are STARK proofs (FRI) and keep their own grind.

• It ports WHIR recursion on GPU (RPX) with ZisK-style proof formats: block 25368371 in 32.62 s #1010's grind 18 (landed there at 2687e48). Opt-out: LAMBDA_VM_ZF_WHIR_GRIND_BITS=20, which reproduces the previous proofs byte for byte.

Security. The calculator of record: Johnson bound, η = 1/300, the fold proof of work unground.

• The binding WHIR phase is the unground fold at 27 variables, 130.393 bits at both settings. Neither Q nor the query grind reaches it.
• The query phases move from 130.926 to 130.907 bits (15 to 27 variables).
• The argue terms and the tree's FRI phases do not move. The proof minimum stays 128.946 under the campaign accounting.
• The grind bits are a verifier constant:
  • a block ground at 18 bits is refused by a verifier at 20, and the reverse;
  • a leaf emitted at 20 refuses an 18-bit block;
  • at chain level, the host and the machine refuse fewer grind bits at the same Q.

Measured behind the knob, on c55dec4, one binary per box:

• 1× (ULTRA 019, 8 + 8 runs after one warm-up):
  • phase B −0.344 s (t −12.9), from the grind falling 0.577 → 0.214 s;
  • whole block −0.27 s (18.71 → 18.44 s);
  • recursion −0.01 s, i.e. unchanged.
• Median 25475471 at i-m4b's cap-128 exploration head (BIG 628, A B B A):
  • phase B −3.48 s;
  • whole block −3.42 s (226.51 → 223.09 s);
  • recursion −0.12 s;
  • both arms verified.
• The leaves' cost. The leaves verify two more queries per chain: +1,041 permutations per three-polynomial group. On these two blocks no leaf table crossed a power of two, and the leaf counts did not change (3 at 1×, 23 at the median).

Bytes. The proof bytes change. Under the fixed trace hash and deterministic grind, the 1× top proof's digest is b9b4826d… at the default and 0da6fea9… under =20. Both trees are accepted (ULTRA 015, at this head).

Known limit. The leaf cap (279,000 permutations) sits above 2^18 LFM_HASH rows, and 18 bits moves every leaf about 1.7 % closer to that doubling.

• On the median, the heaviest leaf's LFM_HASH is 256,549 of 262,144 rows: 2.1 % headroom.
• A heavier block can cross it. That doubles the table, which costs that leaf's prove; it does not affect correctness.

Gates.

• Laptop: zf_format::; the chain refusals and the legacy-bytes KAT; the pins the flip moved; the host block refusal.
• ULTRA 019: the leaf refusal (box tier); the opt-out byte-identical to c55dec4.
• ULTRA 015: the landing sha's identity, above.

One-binary A/B: the block tree vs #1010's epoch tree (FAST 416, noepoch/whir @ 454dad6)

Eight arms E B B E E B B E on the same binary (md5 checked after every arm):

• E is WHIR recursion on GPU (RPX) with ZisK-style proof formats: block 25368371 in 32.62 s #1010's tree through its unchanged record chain;
• B is the block tree under the same environment.

• The block base dropped from 20.98 to 18.50 s with the merge: phase-B argue 5.47 → 3.08 s, from WHIR recursion on GPU (RPX) with ZisK-style proof formats: block 25368371 in 32.62 s #1010's Gruen GKR and no-lift paths.
• The prepared openings cost the base +0.49 s (on 18.67 s vs off 18.17 s, n = 2 each).

Base, block 25368371 on FAST (each run against #1010's epoch base on the same binary)

• Phase B: 9.96 s. Host peak: 43 GiB.
• Every block proof the windowed builder produced verified: 14 of 14 (FAST 363–369). Non-windowed: 6 of 6 (FAST 360–362).

Recursion (W3): proved and verified on block 25368371 (FAST 413)

• Leaves are cut along groups: a group's opening covers all of its tables, so a group is the smallest unit a leaf can verify alone. The tree plan (lfm::whir_block::WhirBlockPlan) runs the host verifier's own statement checks, derives every shape from the AIR and every prepared root from the program, and never reads a proof. It prices each group in permutations and partitions the groups over the leaves (heaviest first, onto the least-loaded leaf).
• A leaf:
  • replays the statement as constants and the roots block over every root of the block;
  • for each of its groups, on that group's fork, runs WHIR recursion on GPU (RPX) with ZisK-style proof formats: block 25368371 in 32.62 s #1010's epoch legs (tables, the group's opening, the prepared openings);
  • publishes the block's id, the transcript state, the public output, and its share Σ p/q of the bus. Leaf 0 subtracts the COMMIT-bus target.
• Nodes are the STARK block's (block_node, shared byte for byte with STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013). They check the id, the state and the output equal across children, add the shares, and the top asserts zero.
• Final check: verify_block_tree(elf, statement, top) derives the top program and checks the top proof against it. Every preset is pinned inside: the base's options, the block format, the tree's options, the leaf rule and the fan-in.
• Tests:
  • every new check has a negative, and each negative a mutation showing that check is the one refusing;
  • the leaf's challenges equal the host's;
  • a tree over another partition is refused at the final check.

Every proof was verified by the harness off the clock. The base is unchanged by the emission running beside phase B (20.98 s vs 21.18 s in the arms without the change).

Next:

• one prepared stack per group (group 8 carries 12 prepared chains);
• the base levers: build_traces over segmented lists, and KECCAK_RND streamed.

Base levers after the A/B (FAST 417–422, each S/P on one binary)

Memory (D-MEMORY, with i-mem / i-mem2)

Narrow trace storage (M4, lane i-m4)

Prover-only; the proof bytes are the same. The proof digest equals the head's before M4 (07d1bd43…) under the fixed trace hash + deterministic grind (FAST 820).

• What changes: between phase A's commit and phase B, each table of at least 2^16 cells is kept on the host packed at the width its values need, 1, 2, 4 or 8 bytes a column (BlockOptions::narrow, production Narrowing::CARD).
  • The pack runs on the card from the group's resident store, on a thread of its own, after the group's commit.
  • Phase B uploads the packed table and widens it on the card. A host reader widens on the host.
  • A wrong width map is refused by the kept-tree check (test plus mutation).
• Time-neutral: whole block −0.04 s at 1×.

Max RSS, wide → narrow (BIG 560; the traces themselves shrink about 4×, e.g. 24.89 → 6.20 GiB at 1×):

Before M4, #1014 ran out of memory at 3.03× (i-m4); the narrow line puts the 120.7 GiB edge near 5.5–6×.

Phase A uploads the next group beside the commit (3′, lane i-noepoch-w2)

Prover-only; the proof bytes are the same (digest 07d1bd43… at the default, FAST 832; the commits, their order and their bytes are unchanged).

• What changes: phase A used to upload, commit and retire each group in turn, so the card waited on every group's upload. Now the current group's commit runs on a thread of its own while phase A's thread takes the next group and uploads its columns (BlockOptions::upload_ahead, on by default; BLOCK_WHIR_UPLOAD_AHEAD=0 is the control).
  • The upload starts only after the commit has asked the card for its room, so it takes what the ledger has left; a store the ledger refuses goes up after the commit, as before (no store or commit room was refused in any run).
• Time (FAST 832, A B B A on one binary): base −0.60 s, phase A −0.66 s, whole block −0.61 s; 46 % of the upload seconds hidden. Groups 4–5 cannot hide theirs: the inline layout thread closes them after the previous commit ends, which is the next term (E3, next section: on since 59e9890).
• Memory: one more group's columns held at the peak — the previous group's wide columns and its packed bytes wait for its pack's install while the current group commits (FAST 834, BLOCK MEM terms). At most one group (≤ ≈ 3.8 GiB), constant per group, not growing with the block; max RSS +1.8 to +2.3 GiB at 1× (FAST 832, 833, 834).

Streamed chunks laid out on three threads (E3 on by default, lane i-noepoch-w2)

Prover-only; the proof bytes are the same (digest 07d1bd43… at the default, FAST 838 and 835; the groups, their order and the packing are unchanged).

• What changes: layout_workers 0 → 3 in BlockOptions::production (59e9890). The streamed chunks are laid out on three threads, with at most K + 1 = 3 of them unpacked at a time (layout_ahead, E3), instead of on the builder's one layout thread. BLOCK_WHIR_LAYOUT_WORKERS=0 (the inline layout, 01da99f's) is the control.
• Why it pays now: with phase A uploading ahead (3′), the inline layout closed groups 4–5 only after the previous commit had ended, so the card waited on them; three workers close them in time.
• Time (FAST 838, A B B A on two binaries, A = 01da99f, B = 59e9890): base −0.54 s (t −14.7), phase A −0.58 s, whole block −0.48 s (t −12.4; 20.23 → 19.75 s); phase B +0.06 s (one job, below what a single job resolves). FAST 835 (one binary, workers 0 vs 3) read whole −1.00 s.
• Memory: max RSS −1.51 GiB at 1× (31.99 → 30.48 GiB, FAST 838; 835 read −0.68 GiB). The bound is what the earlier revert lacked: unbounded, the workers' host memory grew with the block (BIG 390); bounded, it stayed flat at 1.0 / 1.3 / 1.8× (BIG 392, measured before M4 and 3′, not re-run on this head).
• 59e9890 also caps a readout stamp (an upload's "paid" seconds at the upload's own length); no proving change.

Phase B's kept-top paths re-hashed in parallel (lever 1, lane i-noepoch-w2)

Prover-only; the proof bytes are the same (digest 07d1bd43… at the default, FAST 840 at 733557a — 82f9046 adds only the knob; the paths, their order and the refusal are unchanged).

• What changes: phase B serves each revived commitment's first-round paths from the kept tree top. The leaves under each queried block are gathered from the card, re-hashed on the host, and the block's subtree root is checked against the kept node. That re-hash ran one block at a time on the prover's thread while the card waited: 27 gaps of ≈ 37 ms, 1.01 s at 1× (nsys timeline, FAST 839). The blocks are now re-hashed in parallel and kept in block order; the root check still refuses before any path is returned. BLOCK_WHIR_REHASH_SERIAL=1 restores the serial re-hash (a measurement knob).
• Time (FAST 843 + 844 pooled, 6 + 6 + 6 runs: one binary with the knob, serial S vs parallel P, plus 59e9890's binary as the reference R): re-hash 1.01 → 0.07 s; phase B −0.96 s (P − R; P − S −0.93 s, t −30.5); whole block −0.82 s (P − R, t −5.5; 19.72 → 18.90 s); phase A unchanged (this build vs R +0.09 s, t +1.0); max RSS unchanged.
• FAST 840 (two binaries, 2 + 2 runs) read phase A +0.71 s, from one run that stalled in the layout; the code runs only in phase B, and the one-binary check shows no phase-A effect.
• Also in: BLOCK OPEN SPLIT (with LAMBDA_VM_BASE_SPLIT=1), phase B's openings by host stage and the kept-top gather and re-hash seconds.

The tree's leaves execute while their artifacts are built (R2-i, lane i-noepoch-w2)

Prover-only; the proofs are the same (the top proof's digest equal with and without it, 0da6fea9…, under the fixed trace hash + deterministic grind, FAST 841).

• What changes: WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014's tree is proved by the W3 harness (prove_tree_pipelined, the way a prover would run it; there is no production tree driver yet). Its level 0 built all three leaves' artifacts first (three serial card holds, 0.24 s at 1×) and only then let the leaves execute and fill their traces (0.46 s until the first was ready) while the card sat idle (nsys timeline, FAST 839). Execution and fill do not read the artifacts; only the prove does. lfm_prove is now cut into its host half (lfm_execute_and_fill) and its card half (LfmFilled::prove, which asserts the artifacts' hasher is the one the traces were filled for), in the same order; the harness builds the artifacts on a thread of their own and each leaf executes and fills beside them, waiting for its artifacts only to prove. W3_EXEC_BESIDE_ARTIFACTS=0 is the control (the order before).
• Time (FAST 841 + 842 pooled, 8 + 8 runs on one binary): level 0 −0.24 s (t −17.8), recursion −0.23 s, whole block −0.24 s (t −5.2; 18.78 → 18.54 s; per job −0.29 / −0.20); base and max RSS unchanged.
• The first leaf now takes the card 0.48 s after the tree starts, not 0.76 s (one run of each arm, FAST 841's card-hold trace).

A second whole-block field. The W3 readout prints, on the clock between the base and the tree, the block report, a second block frame for the LT heights and the group tables: 0.13–0.14 s at 1× that a prover would not spend. "Whole block" keeps its meaning (every number above includes them); from d169edd on, W3 RECURSION also prints "whole excl. harness readouts" beside it (at d169edd, FAST 841 + 842 pooled: whole 18.54 s, whole excl. harness readouts 18.40 s).

The rest laid out in waves; KECCAK_RND built as its tables (W, lane i-m4b)

Prover-only; the proof bytes are the same (digest 07d1bd43… under the fixed trace hash + deterministic grind: FAST 854 at 0bcc1fc, and FAST 855's W arm on this head's code).

• What changes:
  • The tables the finish builds (the "rest") were turned into columns all at once, and a table's column copy exists before its rows are freed. So for a few seconds most of the rest was held twice, and WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014's memory log put the 1× and 4.13× peaks there (BIG 561). The rest is now laid out in AIR order, in waves of at most 2 GiB of rows, each wave in parallel (BlockOptions::rest_layout_bytes). The tables, their order and the groups are the same (test).
  • The finish built KECCAK_RND as one table of 1,480 columns, which split_keccak_rnd then copied into its 2^16-row tables. At 4.13× that copy was a second peak of the same height. The finish now builds the 2^16-row tables directly (BlockOptions::finish_keccak_rnd_chunks), the same tables as the split's (test), and hands none out during the windows, which would change the groups.
  • BLOCK_WHIR_REST_LAYOUT=all and BLOCK_WHIR_KR_FINISH_CHUNKS=0 restore the old behaviour.
• Memory (BIG 562, one binary at 0bcc1fc, which 70eee3e merges with WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014's later commits; A = both off, W = this; three layout workers; jemalloc never purges, the record posture):

• The live peak (jemalloc active) at 4.13× fell 72.9 → 63.7 GiB and now sits at the finish's end.
• Not as pre-registered, on magnitude: the bands were ≤ 76 GiB and a ≥ 10 GiB drop at 4.13×, and ≤ 27 GiB at 1×. The resident-set gain came from the KECCAK_RND copy alone. The waves bound the live column copies, but the resident set still grows ≈ 16 GiB during the layout, because the freed row buffers stay with the allocator while the column copies take new pages. That growth and the finish's 8-byte tables are the next cut's target (finish packing, being measured).
• On the max-RSS line, the 120.7 GiB edge moves from ≈ 5.9× to ≈ 6.4–6.7× (a linear estimate).
• Time (FAST 852 + 853 pooled, one binary per job, three layout workers): base −0.14 s against both off, phase A −0.12 s.
  • The time comes from the finish ending without the KECCAK_RND copy.
  • The waves add ≈ 0.3 s to the rest's layout, which phase A does not wait on.
  • Packing the rest as it is laid out (pack_rest_as_laid_out) was re-measured with the waves: no gain (+0.01 s), so it stays off.
• Also in:
  • the block memory log, LAMBDA_VM_BLOCK_MEMLOG=1 (off by default). It prints the host memory term by term every half second and at each phase mark, with the line at jemalloc's active peak and each arena's bytes (those two in the lib tests, which install jemalloc).
  • the readouts BLOCK REST TABLES (each rest table's layout end and group) and each group's commit start (from@).

The tree's first finished leaf executes during phase B (R2-ii, lane i-noepoch-w2)

Prover-only; the proofs are the same (base digest 07d1bd43… and the top proof's digest equal with and without it, 0da6fea9… — FAST 841's — under the fixed trace hash + deterministic grind: FAST 845, and FAST 847 again on fac261f, the merge with W).

• What changes: phase B now hands each group's share of the proof (its argue, its opening, its prepared opening) to an observer as that group's opening ends (block_prove_on_forks_observed; the old entry point passes a no-op, and the proof is the same with any observer). A leaf's arena is built from its groups' words alone (group_arena_words + leaf_arena; block_leaf_arena is built from them). The W3 harness turns the groups into words as they arrive; the first leaf whose groups are all opened while another group is still to come (leaf 0 = groups 1, 5, 6 at 1×, done after group 6) executes and fills its traces on a thread of its own, about 1.5 s before the base ends, and the tree picks it up. Off the clock its arena is checked against the finished proof's. W3_LEAF_DURING_PHASE_B=0 is the control.
• Time (FAST 845 + 846 pooled, 8 + 8 runs on one binary, at c5d9cec before the merge with W): whole block −0.24 s (18.63 → 18.39 s; per job −0.21 / −0.28); phase B +0.02 s; max RSS unchanged. The card's wait before the first leaf's prove fell from 0.23 s to 0.003 s; level 0 gained 0.15 s of it, since the first prove now runs beside the other two leaves' execution and fill.
• Memory: the host's high-water is set in phase A at every size (1×: 25.5 GiB active at its peak, 7.2 GiB active by group 6's end with 31.0 GiB resident; 4.13×: 63.7 GiB at the finish's end, i-m4b), so one leaf's traces during phase B's tail fit in what phase A already holds.

The finish's tables packed as they are built (b2, lane i-m4b)

Prover-only; the proof bytes are the same (digest 07d1bd43… under the fixed trace hash + deterministic grind at both arms: FAST 855 at 0cebe16, and FAST 857 again on 365e3ab, the merge with R2-ii).

• What changes:
  • After W, the finish still built its tables 8 bytes per cell, and phase A transposed each one into columns before packing it on the card. At 4.13× that layout took 24.6 GiB of wide tables and grew the resident set by ≈ 16 GiB (BIG 562).
  • The finish now packs each table into the narrow layout as soon as it is built (BlockOptions::pack_finished). KECCAK_RND is packed in waves of four 2^16-row tables. Phase A takes those tables as narrow columns and uploads them as they are, so the card packs only the groups' other tables. Nothing packed is transposed.
  • KECCAK, ECSM and ECDAS stay wide (BLOCK REST PACKED counts the rest: 187 of 191 tables at 4.13×).
  • A packed table's preprocessed columns are checked word for word against the AIR's before it is committed; a wrong one is refused (test, with a mutation).
  • BLOCK_WHIR_PACK_FINISHED=0 restores W.
• Memory (BIG 563, one binary at 0cebe16; O = W and b2 off, A = W, B = W + b2; the memory log on; jemalloc never purges):

• At 4.13× the live peak (jemalloc active) fell 63.7 → 49.7 GiB. The resident set now sits within 2 GiB of it (retention +2.0 GiB, from +14.7), and the rest's layout adds +1.15 GiB to it instead of +16.1. Every pre-registered row is in but one: the 1× live peak, at 21.0 GiB against a band of ≤ 19, because at 1× it falls in the middle of the finish, where the commit pipeline's in-flight copies bind.
• On the max-RSS line (52.10 GiB at 4.13×, 12.82 + 9.47 GiB per 1×), the 120.7 GiB edge moves from ≈ 6.4–6.7× to ≈ 11.1–11.4× (a linear estimate for the base alone; the median block is 9.78×).
• Next in line at the 4.13× peak: the committed groups held narrow for phase B (18.5 GiB, ≈ 5.9 GiB per 1×), then the finish's tables still being built (16.5 GiB).
• Time (FAST 855 + 856 pooled, 8 + 8 runs on one binary): base −0.36 s (t −6.2; per job −0.40 / −0.32), all of it in phase A: the rest is laid out in 0.13 s instead of 0.57.

Measurement posture: the card's pool now retains freed memory (baseline shift)

Not a prover change; the proofs are the same (the top proof's digest 0da6fea9… under both postures, fixed trace hash + deterministic grind, FAST 848; 351a773 adds only a test readout).

• What the env set, and why: every WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014 box run (the canonical env since FAST 416) set LAMBDA_VM_MEMPOOL_RELEASE_MB=0. The card's stream-ordered memory pool then hands its freed blocks back to the driver at each synchronize, so a VRAM sampler's total − free reads the live working set. The code default is to retain every freed block (DEFAULT_MEMPOOL_RELEASE_THRESHOLD_BYTES = u64::MAX in math-cuda), and that is what a prover runs.
• What release-0 cost: each synchronize paid for the release, and the next large allocation mapped the memory again. In FAST 839's trace, the time spent inside cuStreamSynchronize and cuMemAllocAsync while the card was idle was ≈ 0.25 s in phase A, 0.42 s in phase B (0.26 s of it in the encode: one 18–33 ms gap per group) and 0.34 s in the tree.
• Measured (FAST 848 + 849, one binary at 351a773, release-0 vs the default, 8 + 8 runs): whole block 17.91 → 17.27 s, −0.64 s (t −6.0). Phase B −0.44 s (t −28, against the 0.42 s above); phase A −0.07 s (t −0.7; phase A's runs scatter); recursion −0.12 s (level 0 −0.09 s); base −0.51 s; max RSS −0.16 GiB.
• Gates, every run: the tree verified; the log's posture line matched its arm; no host fallback, device commit error, decline or ledger refusal; the card's peaks 24.15 GiB (the ledger's reservation) and 22.8 GiB (the pool's live high-water), read from the ledger and the pool, not from total − free (the new W3 DEVICE line).
• From 351a773 on, WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014's numbers are taken at the code default. Every earlier number in this description was taken under release-0 and stays as measured. VRAM figures come from the ledger or the pool's high-water, never from total − free.

The prover refuses a partition over the group maximum (lane i-m4b)

Prover-only; the proof bytes are the same (digest 07d1bd43…, FAST 858). The prover now refuses a block over BlockFormat::max_groups as its groups close, with the verifier's own InvalidTableCounts error, instead of proving a block the verifier refuses. The median block (90 groups against the cap of 64) spent its whole ≈ 190 s base on such a proof (BIG 564). A test covers both prover paths, with a mutation for each check.

The tree's nodes built beside it, each level proving as its nodes arrive (A, lane i-noepoch-w2)

Prover-only (the W3 harness); the proofs are the same (the top proof's digest 0da6fea9… — FAST 841's — with and without it, under the fixed trace hash + deterministic grind: FAST 889, and FAST 891 again after the permit fix below).

• What changes:
  • The W3 tree (prove_tree_pipelined) built every node program and its artifacts on one thread, level after level, and level 0 joined that thread before any node proved, so level 1 waited for the whole tree's builds. At the median block, 32 s of serial node builds after the leaves' artifacts held level 0 open 10.7 s past its last leaf proof, and the card sat idle 11.4 s before level 1 (BIG 565 / 568's card trace). At 2.66× the wait was 1.5–2.1 s (FAST 888).
  • Now a builder runs beside the whole tree and puts each node's program and artifacts in a slot of its own as soon as they exist. A level's programs are emitted together on a pool of W3_EMIT_THREADS threads (default 4, the builder's own: on the global pool a prover's join can steal an emission and leave the card idle, STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013's FAST 454), and the builder's thread builds each one's artifacts as it arrives. Each node proves once its children are proved and its slot is filled. A builder that stops fails every empty slot. W3_NODE_PIPE=0 is the control.
  • This is STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013's slot pipeline (i-tree's per-node emission, −7.32 s at the median there) in the W3 harness, with three differences: W3's Published<T> serves as the slot; there is no per-group early emission (W3 builds every leaf's artifacts up front, so a level's programs are emitted together); and there is no host-only thread marking (WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014 has none).
  • The level builder is generic (build_levels, an emit on the pool and a finish on the builder's thread). A toy tree at the median's shape puts the serial builder's programs in their slots whatever order a pool finishes them in, and finishes every node off the rayon workers (test; finishing on a worker, or publishing in arrival order, fails it). A failed build fails every slot above it without a hang (test).
• Time:
  • Median (BIG 611, block 25475471 at i-m4b's cap-128 exploration head with these commits, P O O P on one binary): whole block 242.69 → 233.80 s, −8.89 s, every pipeline run below every control run (the estimate from BIG 565 / 568 was −10.6 s). Recursion 52.85 → 43.89 s; level 1's start gap 12.52 → 1.73 s (card idle 11.4 → 1.5 s). Peak VRAM 24274 → 24242 MiB and VmRSS 112.44 → 111.95 GiB: no growth (the gates).
  • 2.66× (FAST 889 + 890 pooled, one binary, 4 + 4 runs): recursion −1.22 s (t −5.2; whole block −1.21 s). Level 1's start gap (the last leaf proof's end to level 1's first prove on the card) fell from 1.97 to 0.69 s. After the permit fix: −1.19 s (t −6.5, FAST 891).
  • 1× (8 + 8 runs): recursion +0.007 s (t +0.3). At 1× the tree has one node, built in the same 1.8 s either way, so there is nothing to take.
• What is left: a level still starts only after its children's proofs and its node's own execute + fill (0.45 s at 1×, 0.63 s at 2.66×; at the median the card sits idle 0.9–1.1 s at each later level's start). That start term is the next one; it is not addressed here.
• Part of this landing: the card permit's latent re-entry, closed (lane i-m4b's diagnosis and repro). W3 built its leaves' artifacts as rayon jobs that each held the card around a build that uses rayon. A holder that waits inside rayon runs queued jobs on its own thread, so a sibling build could take the card a second time on that thread: the permit's assert fired and BIG 569's run parked. Which run trips it was the scheduler's choice (568 ran clean on the same code). Now:
  • the armed permit refuses a hold on a rayon worker before it takes the card, so any hold inside a rayon job fails at once instead of by luck (i-m4b's repro test now asserts the refusal; a plain thread holding the card while rayon builds under it is tested too);
  • W3 builds the leaves' artifacts one after another on their own thread (each held the card throughout anyway);
  • the node pipeline builds artifacts only on the builder's plain thread;
  • the harness disarms the permit before its off-the-clock verify, whose WhirBlockPlan::programs builds as rayon jobs.
• Also in: W3_NODE_TIMES=1 (off by default). It prints, per program, when it was built, taken and proved; its prove's split (execute, fill, the prove net of the card wait, the wait); and each node level's start against the level below. The stamps are taken either way and cost a handful of Instant reads.
• Toward a production tree driver: WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014 has none, and prove_tree_pipelined is its de-facto driver. With the slots it now has a driver's shape: leaves from phase B, nodes from slots, levels as their nodes and children arrive. Moving it out of the test harness is a landing item for STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013 and WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014 both.

Each node proved as soon as its own children are (C, lane i-noepoch-w2)

Prover-only (the W3 harness); the proofs are the same (the 1× top proof's digest with and without it, under the fixed trace hash + deterministic grind, both trees accepted: 0da6fea9…, BIG 614 — the digest FAST 841 / 889 / 891 printed).

• What changes:
  • After (A), a level still started only once its whole level below was proved, and then ran its first node's execute + fill with the card idle. At the median the card sat idle 1.4–1.6 s before level 1 and 0.9–1.1 s before each later level (BIG 611).
  • Now one pool of siblings workers proves every program of the tree, leaves and nodes, taking them in topological order (the leaves, then each node level) and publishing each proof into a slot of its own. A node waits only for its own children's slots, so it executes, fills and proves while the rest of the level below still proves. W3_DATAFLOW=0 is the control: each node waits for its whole level below (tested never to start early).
  • Workers that take programs in this order reach a node only after every program before it has been taken, so the earliest unfinished program never waits on one not yet taken: no deadlock at any number of workers (test; taking the nodes first deadlocks it). The siblings bound, and so the memory in flight, is unchanged. A failed or panicking prove fails every program above it without a hang (test).
  • A level's W3 LEVEL wall now runs from the level below's last proof to its own.
• Time (one binary, arms alternated on one box):
  • Median (BIG 612, block 25475471, F O O F): recursion 43.78 → 42.05 s, −1.73 s, every dataflow run below every control run. The card's idle at level 1's start fell 1.4 → 0.0 s, and at level 2's 0.93 → 0.0 s, in both runs. Whole block −2.60 s (234.15 → 231.55 s), but −0.87 s of that is the base, which the knob cannot reach (it is read after the base): recursion is the number. VRAM flat (24306 → 24274 MiB); the tree phase's VmRSS max 120.05 GiB (one control run) / 112.07 GiB.
  • 2.66× (BIG 614 + 615 pooled, 6 + 6 runs): recursion 12.51 → 12.02 s, −0.49 s (t −4.5). Whole block −0.13 s (t −0.8): the base read +0.37 s (t +2.1), all of it in phase A's host front (executor and trace build), before the knob is read. Phase B was equal within 0.02 s, so nothing moved out of it. At 1× the base moved +0.04 s.
  • 1× (8 + 8 runs): recursion −0.02 s (t −0.5; per-run sd 0.08 s). At 1× the tree has one node, so there is nothing to take.
• Mechanism, as registered (2.66× level-1 start, card idle with no hold active, ≤ 0.30 s): 0.31 s, OUT by 0.007 s (control 1.11 s).
  • Four of the six dataflow runs read 0.00. In the other two (0.88 / 0.96 s), the first level-1 node ready to prove had its program by 4.0 s, but its artifacts (a 0.14 s build on the card) waited 2.0 s behind level 0's proves and were ready only after level 0 ended. A node's slot holds its program and its artifacts together, so its execute + fill waited too, and ran with the card idle. In the four other runs the same build got the card between two leaf proves.
  • So the residual is the slot waiting for the artifacts before the execute, not the dataflow. At the median, level 1's idle read 0.00 in both runs. (Corrected: the first version of this section blamed the builder's emission; the build stamps include the card wait.)
• A readout note: under dataflow the levels overlap, so a level's start gap (from the level below's last proof to the first prove hold starting after it) can count the next level's own proves. At the median, level 2's gap read 0.65 s with the card idle 0.00 s (BIG 612). The level starts are therefore read as card idle with no hold active.
• What is left: the top node's execute + fill after its last child (1.0–1.1 s of card idle at the median, BIG 612). That is (B)'s target, next.
• Toward a production tree driver: with (C), the driver's prove loop is one topological pool over the slots, with the levels taken from the plan's shape.

The held tables spill to disk, by default only when memory is short (lane i-m4b)

Prover-only; the proof bytes are the same: one shape and one proof digest (07d1bd43…) across the base and all four policies, under the fixed trace hash + deterministic grind (BIG 616). The 1× top proof's digest (0da6fea9…) and the tree verify at the landed merge (ULTRA 006).

• What changes:
  • In phase A, after each group is installed, the narrow (packed) tables of every committed group past the first two can move to STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013's spill store. The store (crypto/stark/src/spill.rs) is imported byte-identical; it digests each slot and writes it with O_DIRECT on a writer thread, off the committer's path. A table moves only when the writer's queue has room at that moment, so the committer never waits.
  • Phase B reads the tables back ahead of their upload, within a window of two groups' bytes, and checks every slot's digest. A table whose columns are still out when its group is read is refused (SpillFailed). After a group's opening its packed columns are let go.
  • The policy is STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013's knob and rule: LAMBDA_VM_BLOCK_SPILL = auto (the default) | off | always | <GiB> (a resident budget). auto spills a table when the host's bytes plus a reserve plus the table would pass the target.
    • Target: LAMBDA_VM_BLOCK_SPILL_TARGET_GIB, else min(the cgroup's limit, v2 or v1; MemTotal) − 10 GiB.
    • Host: max(VmHWM, the cgroup's charge − its inactive file pages), the working set. Copied from STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013 (cf253d2, landed at 035aef5), so page cache the kernel reclaims first does not count.
    • WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014's reserve: 1.65 GiB per G cells committed + 6 GiB. It covers the finish's transient, the tree's leaf programs alive through phase B, and phase B's bump with the read-back window. Inferred from BIG 565 (the median without spill).
  • BLOCK SPILL prints the policy, the slots and bytes, the writer's split, the read-back, and the largest host reading auto decided on.
  • Tests:
    • the same bytes spilled and held;
    • a byte flipped in a spilled slot is refused, and so is a slot that does not come back;
    • a budget the block fits in spills nothing, and a zero budget spills every table past the first two;
    • unit tests of the knob, the rule, the cgroup v2/v1 readers and the working-set measure, each with a mutation that fails it.
• Measured:
  • 4.13×, always (BIG 566, single runs): 22.74 GiB spilled, max RSS 52.53 → 40.12 GiB, whole block +1.15 s, drivers waited 0 s.
  • 4.13×, auto (BIG 567): by default nothing spills (target 110.7 GiB). Under a 35 GiB target it spills 21.21 GiB, max RSS 41.04 GiB, +2.91 s.
  • The median, on the cap-128 exploration head (BIG 568; the cap is not part of this landing):
    • auto spilled 26.98 GiB from group 44 of 90, and the tree verified.
    • VmRSS fell only 110.55 → 105.32 GiB. Under the never-purge allocator posture, freed pages stay resident, and the whole-run peak forms in phase A's finish, which the spill does not reach.
  • The landing gate (BIG 616 at 3d99c64, base WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014 54aae28, two jobs pooled, arms B, D = auto, O = off, S = always, T = auto at a 47.5 GiB target):
    • the card tests;
    • one shape and one proof digest across all five arms;
    • every auto run spilled 0 in both jobs, at 110.7 GiB and under the 47.5 GiB target (FAST's limit, kept as a real test on BIG's 128 GiB; the host read 25.3–26.8 GiB across those runs);
    • every run's environment the same size;
    • D − B −0.063 s (t −0.6), O − B −0.048 s (t −0.4), both inside the box's ±0.23 s band.
  • Earlier 1× gates on FAST (880 + 881, 882): always and auto at 0 spilled within noise.
  • ULTRA 006 at the landed merge 0d61081: 1× top digest 0da6fea9…, tree accepted, auto spilled 0 (host at most 27.65 GiB).
• What is left: the median's peak is phase A's own (the finish's transient over the tables still held when it starts), not phase B's. That is the next memory term.

Phase 4 counts the BITWISE lookups in slices (#1013's c369843, lane i-m4b)

Prover-only; the bytes are the same:

• the 187 whole-run trace digests (four programs at the small and default caps) are identical with the slices on and off;
• the 1× proof digest is 07d1bd43… at both, under the fixed trace hash + deterministic grind (ULTRA 010).

What changes:

• The finish's phase 4 counts the BITWISE multiplicities in up to 8 parallel histograms.
  • The in-walk lookups and MEMW_R were already cut into slices. Every other source was one collector that first built the whole list of lookups it sends: LT 8 per op, KECCAK about 5,000 per permutation, then MUL, DVRM, BRANCH, SHIFT and the rest.
  • At the median, those lists, built side by side, were the largest term of the peak that does not spill (unnamed heap 7.5 → 31.8 GiB inside phase 4, BIG 568).
• Every source that is a sum over its ops is now cut into slices of whole ops. MUL and DVRM, which deduplicate per instance, get one slice per instance.
  • This is STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013's c369843, in STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013 since 10-02 and cherry-picked here. WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014's trace builder had diverged from STARK no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1013's and never got it.
• LAMBDA_VM_P4_SLICED=0 keeps the whole-source collectors. The knob is WHIR no-epoch (prove-and-retire) prover: block 25368371 — work in progress #1014's only, for the A/B.
• The slice lengths come from one function.
  • A test counts MUL, DVRM and LT whole and slice by slice, and the counts must be equal.
  • Its MUL and DVRM ops repeat inside instances, so a cut one row off changes the counts (the test checks that); a slice length one row off fails it (mutation, MUL and DVRM).

Measured: the median, BIG 619. One binary on the cap-128 exploration head, N P P N, auto spill on. N runs the whole-source path, P the slices.

• Phase A −9.5 s (whole block −9.3 s) comes from the finish, not the spill: every run spilled ≈ 25–26 GiB.
  • Phase 4 lost its two long poles (LT, KECCAK), so the finish builds the run 8.5–11 s sooner.
  • Phase A's wait for the rest's groups falls 33.8 / 32.0 → 20.8 / 22.6 s.
• 1× (ULTRA 010, 8 + 8, one warm-up discarded): base −0.30 s (t −2.0), recursion −0.01 s.
• The pre-registered bands for phase 4's transient (≤ 9 GiB), the active peak (≤ 83) and phase A (±3 s) were missed, all on the good side. Landed on the lead's judgment; there is a row in FAILED-LEVERS.
• Proves 25416971 (12.4×) at cap 128, exploration (BIG 621, acfd54a + the cap, defaults): 40.7 G cells, 109 groups, base and tree verified; whole-run VmRSS 98.0 GiB, cgroup 106.2; 40.3 GiB spilled; whole block 278.1 s. On the line through the median and this block (≈ 1.18 GiB per G cells), cap 128 binds first, at ≈ 14.4× (≈ 47.8 G cells); memory binds at ≈ 15.5–16×.
• What is left: the median's active peak is now the finish's phase 5. It holds the rest's tables until the finish ends (unnamed +24.0 GiB), beside the held narrow tables (33.8 GiB), which auto spills only once memory is short.

The top executes each child's share as that child is proved, while a child is still unproved (B and B′, lane i-noepoch-w2)

Prover-only (the W3 harness and the LFM executor); the proofs are the same. The 1× top digest is 0da6fea9… with the streamed top on and off, under the fixed trace hash + deterministic grind, and both trees are accepted (BIG 617; ULTRA 008 and 009; ULTRA 011 at c55dec4 itself).

• What changes:
  • The top node executed its whole program after its last child was proved, although most of that program verifies one child at a time.
  • executor::StreamedExecution runs an execution as its arena groups land: each instruction runs in the wave in which the last group it reads lands (each instruction's group set comes from one forward pass), in the level schedule's order, into the slot the schedule gives it. So the witness is execute's, word for word.
  • Tests: the streamed execution is byte-identical in every landing order; a missing group, a group landed twice, or a wrong arena count or length is refused; finish checks that every instruction ran once. Dropping the group propagation (ReadBeforeWrite) or re-running instructions across waves (DoubleWrite) fails them.
  • The W3 top lands each child's arenas as that child is proved (W3_STREAM_TOP, on by default), so only the last child's share and the fill remain after the last child.
  • The readiness gate (B′): the top streams only if a child is still unproved when its program and artifacts arrive; otherwise it executes whole, on the control's path. W3 TOP EXECUTE prints which. A test covers a program ready before the last child (streamed), after it (whole), and a failed child (whole, which then reports the failure). Inverting the predicate fails the test.
• Why the gate: the first version (B) always streamed. At 1× and 2.66× the top's slot arrives after its last child: its artifacts are a short card hold that queues behind the children's proves. So there was nothing to stream behind, and the streamed path costs more than one whole execution. 1× recursion rose by 0.22 s (t +5.8, BIG 617 + 618), and the top's start idle rose too (1.01 → 1.20 s). (B) was not landed. With the gate, the top executes whole at those sizes.
• Time:
  • Median (block 25475471, BIG 613, S O O S O S on one binary): recursion −0.47 s (t −3.2); the card idle at the top's start fell 1.07 → 0.69 s. There the top's program is built 6–8 s before its first child, so it streams. BIG 620 (one gated run at the median) streamed, with recursion 41.91 s, inside 613's streamed range.
  • 1× (ULTRA 009, 8 + 8 after a warm-up): recursion +0.019 s (t +0.8), within the registered ±0.05 s; every gated run executed whole. At 2.66× the top's slot also arrives after its last child, so it executes whole there too.
• What is left: the top's fill and the streamed path's own overhead (≈ 0.7 s of card idle before the median's top). Splitting a node's slot into its program and its artifacts, so that a node executes before its artifacts are built, is the next lever.

A node executes from its program; only its prove waits for its artifacts (b′, lane i-noepoch-w2)

Prover-only (the W3 harness); the proofs are the same: the 1× top digest is 0da6fea9… with W3_EXEC_EARLY on and off, under the fixed trace hash + deterministic grind, and both trees are accepted (ULTRA 012; ULTRA 013 at 806c211 itself).

• What changes:
  • A node's slot held its program and its artifacts together, and its artifacts are a card hold that queues behind the proves below it. So a node could not execute until its artifacts were built, although executing needs only its program. At 1× and 2.66× the top's slot arrived 0.12–0.15 s after its last child, though its program had been emitted 0.03–2.1 s before it. At 2.66× a level-1 node's artifacts waited up to 2.1 s for the card while its children were already proved (BIG 614 / 617 / 618's card holds).
  • Now the builder publishes each node's program as soon as it is emitted: on the pool thread that emitted it, before the builder's thread builds its artifacts. The node executes and fills from the program once its children are proved, and only its prove waits for its own artifacts (node_flow). The traces are filled under the block hasher, which the node artifacts are built for. The top's streaming choice (B′) is taken when the top can execute. W3_EXEC_EARLY=0 is the control.
  • Tests:
    • A toy at the median's shape runs the real builder, scheduler and node flow at 1 to 6 workers, without a hang. It checks that every node gets its program within 40 ms of its emission or its take, executes before its artifacts where it can, and proves only with its own artifacts, after they are built.
    • Mutations fail it: waiting on another node's artifact slot; waiting for the artifacts before the execute; publishing a program after a sibling's finish.
    • The readiness predicate's test, the deadlock test (1–6 workers) and the failure-propagation test were re-run.
  • Part of this landing: proving filled traces against artifacts built for another hasher is refused (LfmProveError::HasherMismatch, naming both) instead of failing a release assert, and the streamed execution's finish returns an error instead of asserting its public words' capacity. Each has a test and a mutation.
• The first build published each program from the builder's thread, between finishes that wait for the card. In one of BIG 622's six runs, a level-1 node got its program 2.3 s after it was emitted, so level 1's idle read 0.183 s against the registered ≤ 0.15. Publishing on the pool fixed it (BIG 624).
• Time (one binary per box, arms alternated):
  • 1× (ULTRA 012, 8 + 8 after a warm-up): recursion −0.274 s (t −15.1; 4.125 → 3.851 s). The card idle at the top's start fell 0.536 → 0.239 s: the top's program now arrives before its last child, so B′ streams it.
  • 2.66× (BIG 624, 6 + 6 after a warm-up): recursion −0.410 s (t −2.8; 12.008 → 11.598 s). Level 1's start idle fell 0.197 → 0.007 s and the top's 0.795 → 0.525 s.
  • Median (BIG 625, N O O N): unchanged. In every run, every node is taken at least 1.27 s after its artifacts exist: the three workers are busy with the 23 leaves. So the split has nothing to act on there. 625 read −0.55 s, but the control arm itself moved +0.52 s from BIG 623 with no change on its path; over both jobs, −0.31 s at t ≈ −1.3. It passed its registered row by chance.
• What is left: the top's fill and the streamed path's overhead (≈ 0.5–0.7 s of card idle at the top's start), and at the median the worker-bound leaf level.